Assisted application operation service for mobile devices using screen sharing

ABSTRACT

A system and corresponding methods to provide assistance to a mobile user ( 42 ) to operate applications in the mobile device ( 31 ) is described. A call center is enhanced to enable routing mobile device application screens ( 43 ) to call center agents in addition to routing voice calls. Such a call center is referred to as remote help center. 
     When a mobile user ( 42 ) needs assistance in operating applications ( 44 ) at the mobile device ( 31 ), he makes a call to remote help center and sends display contents from the mobile device ( 31 ) into the remote help center which routes the display contents to corresponding remote help center agents who can provide remote assistance to operate applications ( 44 ) at the mobile device ( 31 ).

CROSS-REFERENCE TO RELATED APPLICATIONS

Not applicable

FEDERALLY SPONSORED RESEARCH

Not applicable

SEQUENCE LISTING OR PROGRAM

Not applicable

BACKGROUND OF THE INVENTION

1. Field of Invention

The present invention generally relates to assisted applicationoperation services and specifically to assistance provided from a remoteservice center using screen sharing.

2. Prior Art

Application operation assistance is necessary for mobile device userswho want to operate their applications while they are mobile. Withadvent of smartphones, the number of applications in a mobile device hasincreased exponentially. This has lead to a complex user interactionwith several menus and submenus to get to a desired set of features inapplications. Hence it is desirable to have assistance to operate thecomplex set of menus to access features quickly and efficiently.

Technology in smartphones is improving rapidly providing fasterprocessors and increased memory, enabling complex-desktop likeapplications to be executed in a smartphone. But complex desktop likeapplications provide a lot of features and accessing all of thesefeatures without compromising user experience is a big challenge. Hencemobile device users and application designers face an increasingchallenge to balance ease of use with complex features of anapplication. Most mobile devices such as smartphones only provide asmall keypad or a touch sensitive screen which is not sufficient toeasily access complex features of desktop like software applications.

There have been several attempts at providing a better user experiencein mobile devices for accessing features of complex applications. Theseinclude providing a small keypad to enable entering data intoapplications, providing touch sensitive screen with finger gestures toenable quick access to some features and providing a larger display toenable better content viewing experience.

But all these attempts cannot replicate a user experience provided by adesktop or laptop hardware that has a large keyboard, mouse and a largedisplay to enable efficient use of complex desktop application software.

A small keypad in a smartphone can only allow thumb based typing andthat can be very limiting while editing large documents.

A gesture based touch screen may enable faster access to some often usedapplication features, but getting access to several hundred features isimpossible using gestures as a user cannot remember all the gesturesnecessary. Accessing large number of features is only possible byproviding menus and toolbar items in predetermined locations of anapplication.

Providing a slightly larger display helps in viewing content such as aweb page, parts of which may be scaled and panned to provide a betterview of localized content in a screen. But such a system is useful onlyfor applications that are tailored for viewing and not editing.Applications that require a large amount of user interaction cannot beoperated with frequent scaling and panning of a screen.

Another solution to this problem in prior art is to provide a minilaptop called the netbook. Such netbooks provide a reasonable sizedkeypad and screen to enable a user to operate almost all of the featuresavailable in a desktop application. But such devices are too big to becarried by people on person, without having an extra case or a bag forit. These devices cater to a different market segment and are not areplacement for a smartphone device that can be carried in a pocket andoperated by a single hand.

Even with such devices, it is impossible to get access to all thefeatures while being mobile. Being mobile, a person can only use onehand and hence it becomes quite difficult to operate all applicationswith one hand using a netbook.

Hence there is a need for assistance to operate applications for amobile device user from a remote service center. Such assistance can beprovided by a live person further referred to as a helper or an agent atthe service center who is equipped with a desktop computer with a largekeyboard, screen and a mouse and can control and operate theapplications at the mobile device of the user remotely.

Currently there are no methods in prior art that offer a solution tothis problem. It is the intent of present invention to provide asolution to this problem by using screen sharing methods. The presentinvention proposes using screen sharing methods combined with a callcenter like service center that is capable of routing screens of mobiledevice users to corresponding helpers who can remotely control andoperate the screens of a mobile device in addition to routing voicecalls.

Call centers usually route voice calls from a user to appropriate callcenter agent who can handle a call.

The present invention proposes extending call center features by routingscreens of mobile device users to call center agents in addition torouting calls. This particular combination of routing mobile devicescreens to call center agents in addition to routing calls does notexist in any known prior art. Such a service center shall be furtherreferred to as remote help center.

Prior art methods use screen sharing methods to enable multiple users tocommunicate within a group by allowing them to view common data. But allthese users are from a trusted group and such systems cannot be used byindividual users who might need help in entering large amounts of datainto forms or editing a spreadsheet with complex formulae and graphs.

There are several products in prior art that use screen sharingprotocols such as webex, vnc and citrix. But all of these products setupa communication channel among trusted participants and do not provide anindividual mobile device user assistance to operate applications in amobile device.

Most such systems enable a mobile device user to view screens from adesktop user and join a communication session, but none offer a solutionwhere screens from a mobile device are shared and routed to an agent ata service center who can operate all applications at the mobile deviceusing a larger desktop.

Hence it can be seen that there is a need for a service that enables amobile device user to get help in operating applications in a mobiledevice.

Following paragraphs in current section describe relevant prior art inthis field.

Prior art U.S. Pat. No. 6,980,953 proposes a real time transcriptionservice where a user gets documents transcribed from a remote servicecenter. The transcription could be viewed by the user seeking thisservice on a screen in proximity to the user. But this does not addressthe need of a mobile device user to get help to operate applications onthe mobile device. This only caters to a particular need of transcribingdocuments.

Prior art U.S. Pat. No. 7,398,470 proposes a remote printing servicewhere a user is able to view the display of service personnel at aremote service center so that the user can influence design of itemsbeing printed. But this does not address the need of a mobile deviceuser who wants help in operating complex applications at the mobiledevice. This is the opposite of what a mobile device user who wants tooperate applications at the mobile device would want.

Prior art U.S. Pat. No. 6,295,551 proposes a system where a call centeragent and a user share a browsing session. This is achieved by providingan intermediate java applet that communicates changes in browser statefrom a user to call center agent and vice versa. But this is a specificimplementation to provide a joint browsing session where a java appletis provided by the call center agent to the browser of the user. Thismakes use of two different application instances of browser application,one on user device and another on call center agent device. This doesnot transport all screen contents from a mobile device but just showcases how joint browsing can be achieved. This cannot be used forproviding a user with operation assistance for all applications in themobile device. Hence this does not address the need of a mobile deviceuser who wants help in operating complex applications at the mobiledevice.

As can be seen from above, all known prior arts suffer from somelimitations in offering a solution that enables a mobile device user toget help in operating a complex set of applications in the mobile devicefrom a remote service center.

3. Objects and Advantages

Accordingly, several objects and advantages of the present inventionare:

-   -   a) to provide a mobile device user access to a remote help        center from where the user can get help in operating        applications in the mobile device;    -   b) to provide a remote help center that is capable of routing        screens of a mobile device user in addition to voice calls to        appropriate agents who can operate applications in the mobile        device; and    -   c) to provide a protocol that enables efficient routing of        screens of a mobile device user in addition to routing voice        calls to remote help center agents.

SUMMARY

In accordance with present invention a system is provided that enables amobile device user to get help in operating applications in the mobiledevice by a person at a remote service center.

This is achieved by enhancing a call center with screen sharing protocolthat enables routing display screens of a mobile device user to callcenter agents who are live persons who can operate upon applications atthe mobile device remotely, in addition to routing calls.

Combining screen sharing protocol with a call center is unique to thisinvention and provides an unobvious value to mobile device users who canget help in operating complex desktop like applications while on themove.

The enhanced call center is able to route mobile device screens toappropriate agents who can provide the needed assistance. This enhancedcall center is further referred to as remote help center.

Mobile device application that may be operated with remote assistancecould be any application in a mobile device including phone callapplication, calendar application, email application, productivityapplication such as spreadsheet and browser application.

Such applications may be completely executed at the mobile device thatincludes both service logic and graphical user interface, or may beexecuted as a split mode application where graphical user interface maybe executed on the mobile device and the service logic may be executedat a server.

Hence all applications whether running completely at the mobile deviceor in split mode will benefit from this invention in providingassistance to mobile device users in operating complex applications froma remote service center.

DRAWINGS Figures

FIG. 1 shows the system architecture of remote help center.

FIG. 2 shows embodiments of a mobile device.

DRAWINGS Reference Numerals

-   31 mobile device-   32 cellular packet data-   33 internet-   34 mobile call interface-   35 voice switching network-   36 remote help center server-   37 call routing function-   38 screen routing function-   39 helper stations-   40 desktop personal computers-   41 telephones-   42 mobile user-   43 mobile application screen-   44 mobile application-   45 voice path-   46 data connection

DETAILED DESCRIPTION OF PREFERRED EMBODIMENT

FIG. 1 shows the system architecture of remote help center. In preferredembodiment of this invention, mobile device 31 is a smart phone device,with cellular packet data 32 support for connecting to internet 33 aswell as mobile call interface 34 support for connecting to voiceswitching network 35 simultaneously.

A remote help center server 36 provides support for assisted applicationoperation, which includes support for call routing function 37 andscreen routing function 38. Application operation assistance is providedby pool of helper stations 39 that are connected to remote help centerserver 36 through intranet or internet and also through voice switchingnetwork 35. Helper stations 39 are equipped with desktop personalcomputers 40 and telephones 41. Voice switching network 35 can either bePublic Switched Telephone Network (PSTN) also known as circuit switchednetwork, or Voice over Internet Protocol (VoIP). In preferred embodimentof this invention, mobile device 31 is equipped to support complexapplications that are not always convenient to operate with one hand andmay involve complex interactions such as menu navigations, andsignificant data entry.

A mobile user 42 using mobile device 31 initiates assisted applicationoperation service while using mobile application screen 43 of mobileapplication 44 by sending assistance request to remote help centerserver 36 through internet 33. Remote help center server 36 allocatesone of the available helper stations 39 to assist mobile user 42.Assisted application session involves establishing voice connection aswell as application screen sharing session simultaneously between mobiledevice 31 and helper station 39.

In one of the embodiment of this invention, voice connection betweenmobile device 31 and helper station 39 is facilitated using Voice OverInternet Protocol (VoIP). In this embodiment, remote help center server36 performs call routing functions 37 to establish voice path 45 betweenmobile device 31 and helper station 39.

In another embodiment of this invention, voice connection between mobiledevice 31 and helper station 39 is established using Public SwitchedTelephone Network (PSTN).

Remote help center server 36 also establishes application screen sharingsession for assisted mobile application 43 between mobile device 31 andhelper station 39. Screen sharing session is established using a dataconnection 46 through internet 33. Assisted mobile application 43 screenappears in display of helper station 39. Helper using helper station 39can engage with mobile user 42 using mobile device 31 in voiceconversation and also navigate application menu and enter data inassisted mobile application 43 screen in helper station 43. Datamodifications and menu selections of assisted mobile application 43 inhelper station 39 are automatically reflected in assisted mobileapplication 43 screen in mobile device 39.

A mobile application 44 executing at the mobile device 31 can be one ofcall processing application, electronic mail application, calendarapplication, memo pad application, browser application, and remotelyrendered application running at a server.

Mobile application 44 executing at the mobile device 31 can be fullyexecuting at the mobile device 31 or can be executed in a split mode ofoperation.

When mobile application 44 is executed fully at the mobile device 31,both graphical user interface logic and service logic are collocated atthe mobile device 31.

When mobile application 44 is executed in split mode of operation, thegraphical user interface logic is executed at the mobile device 31, butservice logic may be executed at another device other than the mobiledevice 31. Such applications are called remotely rendered applications.

The remotely rendered application could be an application that displaysgraphical output at the mobile device 31, but its service logic maybeexecuting at a different device such as a remote server. Examples ofsuch a system could be applications using X window protocol, or otherprotocols than enable screen sharing such as virtual network computing(VNC).

Routing of voice call and screen data to a specific agent can becontrolled by using agent selection criteria that can be one ofavailability of agents, domain expertise of agents, performance ofagents, geographical location of agents, network latency between saidmobile device 31 and said agent, and language preferences of a user ofsaid mobile device 31.

FIG. 2 shows that mobile device 31 can be one of personal digitalassistant, mobile phone, smartphone, and portable media player.

ADVANTAGES

From the description above a number of advantages of the system thatextends a call center service with screen sharing protocol becomeevident:

-   -   a) a method is provided where a mobile device user gets access        to a remote help center from where the user can get help in        operating applications in the mobile device;    -   b) a remote help center is provided that is capable of routing        mobile device user screens in addition to voice calls to        appropriate remote help center agents who can operate        applications in the mobile device from a remote screen; and    -   c) a protocol is provided to enable efficient routing of screens        of a mobile device user in addition to routing voice calls to        remote help center agents.

CONCLUSION, RAMIFICATIONS AND SCOPE

Accordingly, the reader will see that enhancing a call centerapplication to route mobile device screens in addition to call routingprovides a solution to a mobile device user who can get help inoperating a growing set of complex applications at the mobile device.

Although the description above contains many specificities, these shouldnot be construed as limiting the scope of invention but merely asproviding illustrations of some of the presently preferred embodimentsof this invention. Thus the scope of this invention should be determinedby appended claims and their legal equivalents, rather than by examplegiven.

1. A method of providing assistance to operate applications associatedwith a mobile device from a remote service center further referred to asa remote help center comprising: a) displaying graphical user interfaceoutput of said applications at a display associated with said mobiledevice; b) capturing said graphical user interface output to produce adata stream in a network data representation format; c) sending saiddata stream to said remote help center; d) receiving said data stream atsaid remote help center; and e) routing said data stream to a agent atsaid remote help center further referred to as agent providingassistance, wherein said agent is a person capable of operating saidapplications remotely.
 2. The method of providing assistance of claim 1,further comprising: a) converting said data stream into graphical userinterface elements to form a copy of display contents of said graphicaluser interface output to be displayed at a display associated with saidagent providing assistance; and b) displaying said copy of displaycontents into said display of said agent providing assistance.
 3. Themethod of providing assistance of claim 2, further comprising: a)capturing user interface input events from said agent providingassistance who is using a mouse and a keyboard to generate userinterface input events; b) associating said user interface input eventswith said copy of display contents; and c) transferring said userinterface input events of said agent to said application associated withsaid mobile device.
 4. The method of providing assistance of claim 3,further comprising: a) initiating a voice call to said remote helpcenter and associating said voice call with said mobile device; b)receiving said voice call at said remote help center; and c) routingsaid voice call to said agent providing assistance.
 5. The method ofproviding assistance of claim 4, wherein said voice call is selectedfrom group consisting of voice over internet protocol call, and circuitswitched call.
 6. The method of providing assistance of claim 4, whereinsaid voice call initiation is from a device selected from groupconsisting of said mobile device, device capable of call processingnetworked with said mobile device, and a separate device capable of callprocessing.
 7. The method of providing assistance of claim 1, whereinsaid mobile device is selected from group consisting of personal digitalassistant, mobile phone, smartphone, and portable media player.
 8. Themethod of providing assistance of claim 1, wherein said application isselected from group consisting of call processing application,electronic mail application, calendar application, memo pad application,browser application, and remotely rendered application running at aserver.
 9. The method of providing assistance of claim 1, wherein saidapplications are executed using application execution methods selectedfrom group consisting of complete application execution at said mobiledevice, and partial application execution method at said mobile device,wherein said complete application execution method provides forexecuting both graphical user interface logic and service logic of saidapplication at said mobile device and wherein said partial applicationexecution method provides for executing graphical user interface logicof said application at said mobile device and service logic of saidapplication at a remote server.
 10. The method of providing assistanceof claim 1, wherein said capturing of said graphical user interfaceoutput associated with said application is done at a device where saidapplication service logic is executed.
 11. The method of providingassistance of claim 1, wherein said routing of said data stream to saidagents is dependent on agent selection criteria selected from groupconsisting of availability of agents, domain expertise of agents,performance of agents, geographical location of agents, network latencybetween said mobile device and said agent, and language preferences of auser of said mobile device.